Lecture 5 - General Linear Model (GLM) I
2023-07-25
Scatterplot examples
Null model
If there were NO association, the mean \(\bar{Y}\) would be the best prediction for each student:
Prediction error
This prediction will be a bit wrong for every student:
Link with standard deviation
The mean of those prediction errors (squared) is the variance of “grade”
Its square root is the SD of grade
Association
The points appear to follow a diagonal upward line, rather than the straight line of the mean:
Diagonal line
The distances of points from a diagonal line are obviously smaller than from the straight line of the mean:
Prediction
By following the line you can kind of guess what grade you might expect for a specific number of hours studied. These predictions are better than just using the mean:
Prediction error
We can use the line to predict values of \(Y\) for individuals \(_i\)
- This prediction \(\hat{Y}_i\) is never identical to the observed value of that individual, \(Y_i\)
- There is always some prediction error, \(Y_i - \hat{Y}_i\)
Ordinary Least Squares
We want to obtain the line that gives us the best possible predictions
- There exist values of \(a\) and \(b\) that give us a line with the best possible predictions
- These can be calculated using matrix algebra (not part of this course)
- The resulting line goes exactly through the middle of the cloud of datapoints, which is why \(e_i \sim N(\textbf{0}, \hat{\sigma}^2_e)\)
- This is called ordinary least squares regression
- Will be explained further, but squares refers to squared prediction errors
Numeric example
Substituting numeric values for the coefficients, the function to predict grade based on hours is:
\(\hat{Y}_i = 2.9 + 0.8*X_i\)
Student 71 studies 4 hours, so the predicted grade \(\hat{Y}_{71}\) is:
\(\hat{Y}_{71} = 2.9 + 0.8 * 4 = 6.1\)
In reality student 71’ grade was 8.8, so the prediction error was \(Y_i - \hat{Y}_i = 8.8 - 6.1 = 2.7\)
Visualization
Testing coefficients SPSS
- This tests the intercept and slope
- Default null-hypothesis for the intercept is \(H_0: \beta_0 = 0\)
- Is the intercept significantly different from zero? No.
- Default null-hypothesis for the intercept is \(H_0: \beta_1 = 0\)
- Is the slope significantly different from zero? Yes.
Custom tests using SPSS
Let’s conduct a one-sided hypothesis, \(H_0: \beta_1 \leq 0\)
- Our true belief, in this case, is that the effect will be positive, \(H_a: \beta_1 > 0\)
- If the effect is in the direction of \(H_a\), we can divide the p-value by 2 for a one-sided test
- \(p = .001/2 = .0005\) which we round to \(p < .001\), so no practical difference
Custom tests using SPSS
Or you might wonder: if I would study 0 hours, should I expect a passing grade? \(H_0: \beta_0 \leq 5.5\)
- \(t = \frac{b_0-\beta_0}{SE_b} = \frac{1.31-5.5}{.97} = -4.32\)
- The effect is not in the direction of \(H_a\); instead, it’s congruent with \(H_0\), so we will never reject it
Linearity
- Visual check
- Scatterplot: do the points follow a straight line?
Linearity 2
- Residual plots
Violations of linearity
- Curvilear
- Outlier
Assessing normality
- Histogram, “Normal P-P plot” or “Q-Q plot”
- Kolmogorov-Smirnov / Shapiro-Wilkes test
Normality: Visual inspection
Homoscedasticity plots
